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(54) Multi-point camera system 

(57) A multi-point surveillance camera system uses 
a combination of an image recognition unit (1 02) a zoom 
camera (101) equipped with a pan/tilt driver unit (103) 
forchanging acamera monitor position, to monitor a plu- 
rality of monitor positions. The monitor position is 
changed by using the pan/tilt unit, and an image recog- 
nition unit starts an image recognition process to reliably 



detect an abnormal entrance at the plurality of monitor 
positions, after a focus position adjustment by preset in- 
formation (after completion of commanding to change 
the monitor position to a preset position or stop at the 
preset position), after an automatic focusing operation, 
or after a combination of the focus position adjustment 
by the preset information and the automatic focusing op- 
eration. 
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Description 

[0001] The present invention relates to a camera sys- 
tem having a function of monitoring a plurality of view 
angles and positions and detecting an abnormal en- 
trance from monitored images. 

[0002] A surveillance camera with a fixed monitor po- 
sition for detecting an abnormal entrance by an image 
recognition process is disclosed in JP-A-9-276237. This 
monitor camera always takes an image at a fixed mon- 
itor position and any change in the image is checked by 
the image recognition process to detect an abnormal en- 
trance. 

[0003] Another surveillance camera mounted with a 
pan/tilt driver unit and a preset unit is disclosed in JP- 
A-8-1 39987. This camera is set to take an image at a 
desired position, and pan, tilt, zoom and focus positions 
for that position are preset to control the camera monitor 
position in accordance with the preset values. An image 
recognition method is not disclosed in this publication. 
[0004] There is a high demand that a single camera 
automatically recognizes and watching a plurality of 
monitor positions and view angles. For automatic rec- 
ognition of a monitor object, a recognition camera utiliz- 
ing an image recognition process is known. However, a 
single focus and fixed view angle camera capable of au- 
tomatically monitoring a plurality of different view angles 
and positions is not known. 

[0005] Products of a combination of a tripod head ca- 
pable of panning and tilting and a zoom camera for mon- 
itoring a plurality of view angles and positions are 
known. However, although most of these products pre- 
set monitor positions and angle views to switch there- 
between at high speed and monitor a wide image area, 
they cannot perform automatic recognition of a monitor 
object because an image recognition process is not in- 
corporated. 

[0006] Even if these two kinds of techniques are com- 
bined, image recognition may become unstable each 
time switching occurs among a plurality of view angles 
and positions. Erroneous recognition may occur and an 
abnormal entrance cannot be detected reliably. 
[0007] It is an object of the present invention to pro- 
vide a multi-point surveillance camera system capable 
of obtaining an optimum image recognition result and 
reliably detecting an abnormal entrance even if the cam- 
era is moved to a different preset position or view angle. 
[0008] In order to achieve the above object of the in- 
vention, a combination of an image recognition unit a 
zoom camera equipped with a pan/tilt driver unit for 
changing a camera monitor position is used to monitor 
a plurality of monitor positions. The monitor position is 
changed by using the pan/tilt unit, and an image recog- 
nition unit starts an image recognition process to reliably 
detect an abnormal entrance at the plurality of monitor 
positions, after a focus position adjustment by preset in- 
formation (after completion of commanding to change 
the focus position to a preset position or stop at the pre- 



set position), after an automatic focusing operation 
(hereinafter called an AF operation), or after a combina- 
tion of the focus position adjustment by the preset infor- 
mation and the automatic focusing operation. 
5 [0009] According to another aspect of the invention, 
a combination of an image recognition unit a zoom cam- 
era equipped with a zoom mechanism for changing a 
camera monitor view angle is used to monitor a plurality 
of monitor positions, at least two monitor positions hav- 
10 ing different view angles. The camera monitor view an- 
gle is changed in accordance with preset information, 
and an image recognition unit starts an image recogni- 
tion process to reliably detect an abnormal entrance at 
the plurality of monitor positions, after completion of 
'5 changing to the present position or completion of an op- 
eration by the zoom mechanism, after completion of set- 
ting an in-focus state by the automatic focusing opera- 
tion, or after a combination of the focus position adjust- 
ment by the preset information and the automatic focus- 
20 jng operation. 

[0010] According to the invention, the image recogni- 
tion unit starts the image recognition process after the 
automatic focusing operation, at the same time when 
the automatic focusing operation starts, a timer is acti- 
os vated. if the in-focus state does not enter in a predeter- 
mined time lapse, the focus position is forcibly set to the 
preset position, and thereafter the image recognition 
unit starts the image recognition process to reliably de- 
tect an abnormal entrance at a plurality of monitor posi- 
30 tions. 

In the drawings: 

[0011] Fig. 1 is a block diagram showing the structure 
35 of a multi-point surveillance camera system according 
to an embodiment of the invention. 
[0012] Fig. 2 shows an example of a GUI display 
screen of a controller. 

[0013] Fig. 3 is a flow chart illustrating an operation 
40 up to a start of an image recognition process. 

[0014] Fig. 4 shows an example of an image without 
any intruder. 

[0015] Fig. 5 shows an example of an image with an 
intruder near to the camera. 
45 [0016] Fig. 6 shows an example of an image with an 
intruder remote from the camera. 

[0017] An embodiment of the invention will be de- 
scribed with reference to Figs. 1 to 6. 
[0018] Fig. 1 is a block diagram showing the structure 
50 of a multi-point surveillance camera system according 
to the embodiment of the invention. In Fig. 1 , reference 
numeral 101 represents a camera. Reference numeral 

102 represents an image recognition unit for calculating 
a change in an image by an image recognition process 

55 and detecting any intruder if any. Reference numeral 

103 represents a pan/tilt driver unit for driving a pan/tilt 
mechanism of the camera. Reference numeral 1 04 rep- 
resents a focus control unit for controlling a focus posi- 
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tion in order to adjust the focal point. Reference numeral 
105 represents a preset unit for storing pan, tilt, zoom 
and focus positions and a background image. Refer- 
ence numeral 1 06 represents a control unit for perform- 
ing a pan-tilt control, a zoom control and a preset oper- 5 
ation. Reference numeral 107 represents a zoom driver 
unit for changing the size of a view angle. 
[0019] The operation of the multi-point camera sys- 
tem having the structure as above will be described. 
First, an monitoring operation through pan/tilt driving will 10 
be described. 

[0020] Fig. 2 shows an example of a GUI display 
screen of the control unit 106. As a user manipulates a 
pan/tilt button 201 on the GUI display screen shown in 
Fig. 2, the control unit 106 outputs a pan motion target 15 
position and a tilt motion target position e1 to the image 
recognition unit 102. The image recognition unit 102 
supplies a pan/tilt motion target position a1 to the pan- 
tilt driver unit 1 03. The image recognition unit 1 02 stores 
the present pan/tilt position. The pan/tilt driver unit 1 03 20 
outputs a drive control signal b1 to the camera 101 to 
change the pan and tilt positions and set the camera 
monitor position. 

[0021] For a zoom operation, as a user manipulates 
a zoom button 202, the control device 106 outputs a 25 
zoom position e4 to the image recognition unit 1 02. The 
image recognition unit 1 02 outputs a zoom position h to 
the zoom driver unit 1 07. The image recognition unit 1 02 
stores the present zoom position. The zoom driver unit 
1 07 outputs a zoom control signal i to the camera 101. 30 
The camera 101 drives its zoom mechanism to change 
the size of a view angle of an image. When the camera 

101 stops, the camera 101 outputs a camera motion 
stop signal c2 to the image recognition unit 1 02. An au- 
tomatic focusing (AF) operation is executed by using the 35 
camera 1 01 , the focus control unit 1 04 having the focus 
mechanism and the image recognition unit 102. When 

the image recognition unit 1 02 confirms from the camera 
motion stop signal c2 that the camera motion stopped, 
it outputs an AF activation request signal f 1 to the focus 40 
control unit 104. 

[0022] The focus control unit 1 04 outputs a focus drive 
amount signal d1 to the camera 101 so that the focus 
position of the camera is moved by an amount corre- 
sponding to the focus drive amount signal d1 . The cam- 45 
era 1 01 outputs an image signal d2 to the focus control 
unit 104. In accordance with this image signal d2, an in- 
focus judgement process is executed. The focus control 
unit 104 outputs an focusing operation completion sig- 
nal f2 and a focus position f3 to the image recognition so 
unit 102. The image recognition unit 102 stores the 
present focus position. When the image recognition unit 

102 confirms an in-focus state from the focusing oper- 
ation completion signal f2, the image recognition unit 
102 starts an image recognition process which always 55 
checks a change amount of a monitor image input 
through a lens of the camera 1 01 in accordance with the 
image signal d representative of the monitor image and 



detects an abnormal entrance if any 
[0023] Next, a preset operation will be described. A 
user manipulates a preset number setting button 203 to 
designate a preset number "4". Then, the user manipu- 
lates a preset button 204. The control unit 106 outputs 
the preset number and a preset request e2 to the image 
recognition unit 102. The image recognition unit 102 al- 
ways acquires camera information and outputs present 
camera pan, tilt, zoom (view angle) and focus positions, 
an input image, and a preset number g1 to the preset 
unit 105. The preset unit 105 stores the pan, tilt, zoom 
(view angle) and focus positions, and the input image 
(background image) corresponding to the preset 
number "4" in a memory region for the preset number 
"4". 

[0024] Next, a multi-point monitoring operation using 
the preset data will be described. 
[0025] As a usermanipulates a multi-point monitor ex- 
ecute button 205, the control unit 106 outputs a multi- 
monitor execute request e3 to the image recogn ition unit 
102. The image recognition unit 102 outputs a preset 
data request g2 for a preset number " 1 to the preset 
unit 105. The preset unit 105 outputs preset data g3 to 
the image recognition unit 102. if the preset data does 
not exist, empty data is output. If the preset data does 
not exist, the image recognition unit 1 02 outputs the pre- 
set data request g2 to the preset unit 105 by changing 
the preset number until the preset data is found. If the 
preset number exceeds the maximum preset number, 
an output of the preset data request g2 is terminated. If 
the preset data exists, the image recognition unit 102 
outputs the pan/tilt motion target position a1 to the pan/ 
tilt driver unit 103. The pan/tilt driver unit 103 outputs 
the drive control signal b1 to the camera 101 to change 
the camera monitor position through a pan/tilt operation. 
The image recognition unit 102 also outputs the zoom 
position h to the zoom driver unit 1 07. The image rec- 
ognition unit 102 stores the present zoom position. The 
zoom driver unit 107 outputs the zoom control signal i 
to the camera 101. The camera 101 drives its zoom 
mechanism to change the size of a view angle of the 
monitor image. When the camera 1 0 1 stops, the camera 
101 outputs the camera motion stop signal c2 to the im- 
age recognition unit 102. When the image recognition 
unit 102 confirms from the camera motion stop signal 
c2 that the camera motion stopped, it outputs a focus 
motion target position f4 to the focus control unit 1 04, in 
accordance with the preset focus position. 
[0026] The focus control unit 104 outputs the focus 
drive amount signal d1 to the camera 101 so that the 
focus position of the camera 1 01 is moved by an amou nt 
corresponding to the focus drive amount signal d1 . The 
camera 101 outputs the image signal d2 to the focus 
control unit 104. In accordance with this image signal 
d2, the focus control unit 104 judges whether the focus 
is set to the in-focus state. The focus control unit 104 
outputs the focusing operation completion signal f2 to 
the image recognition unit 102. 
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[0027] When the image recognition unit 1 02 confirms 
an in-focus state from the focusing operation completion 
signal f2, the image recognition unit 1 02 starts an image 
recognition process to detect an abnormal entrance if 
any. If it is judged that the focus is not in the in-focus 5 
state, the image recognition unit 1 02 outputs the AF ac- 
tivation request signal f1 to the focus control unit 104. 
At the same time, a timer starts operating. The focus 
control unit 1 04 starts the AF operation and outputs the 
lens drive amount d1 to the camera 101. The camera 10 
101 changes its focus position by an amount corre- 
sponding to the lens drive amount d1 . The camera 1 01 
outputs the image signal d2. In accordance with the im- 
age signal d2, the focus control unit 1 04 judges whether 
the focus is in the in-focus state. The focus control unit 15 
104 outputs the in-focus operation completion signal f2 
and the focus position f3 to the image recognition unit 
102. The focus control unit 104 outputs the focus posi- 
tion to the camera 1 01 and continues the AF operation, 
until it is judged that the focus is in the in-focus state. 20 
The image recognition unit 1 02 stores the present focus 
position represented by the focus position f3. 
[0028] The image recognition unit 102 judges the in- 
focus state from the in-focus state operation completion 
signal f2. The in-focus judgment by the image recogni- 
tion unit 102 is made in a predetermined time duration 
after the start of the AF operation as counted with the 
timer, and then the image recognition unit 1 02 starts the 
image recognition process to detect an abnormal en- 
trance if any. If the in-focus judgement is not made in 
the predetermined time duration, the image recognition 
unit 102 outputs an AF forcible stop request f5 to the 
focus control unit 1 04. The focus control unit 1 04 stops 
the AF operation. The image recognition unit 1 02 again 
outputs the focus motion target position f4 to the focus 
control unit 104. The focus control unit 104 outputs the 
lens drive amount d1 to the camera 101 . The camera 
101 changes its focus position by an amount corre- 
sponding to the lens drive amount d1 . The image rec- 
ognition unit 102 starts the image recognition process 
to detect an abnormal entrance if any. 
[0029] An abnormal entrance is detected as in the fol- 
lowing. An image difference calculated through a nor- 
malized correlation calculation between the preset 
background image and the present input image, and a 
calculated monitor object area and shape difference, 
are compared with judgement threshold values. If the 
image recognition results are in a range of the judge- 
ment threshold values (object area, shape and so on), 
while the image difference is more than a certain value, 
it is judged that there is an abnormal entrance. After the 
abnormal entrance detection process, the image recog- 
nition unit 1 02 outputs the preset data request g2 for the 
next preset number to the preset unit 1 05 to execute the 
monitor operation for the next preset number and repeat 
the above-described operations. 

[0030] The image recognition process may be started 
after the camera is moved to the focus position corre- 
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sponding to each preset position, without executing the 
AF operation. However, as described above, if the AF 
operation is executed, the camera can reliably follow a 
monitor object (including a moving object) or an obstacle 
in the image to be caused by an abnormal entrance, so 
that a monitor image having a good focusing state can 
be obtained and the image recognition precision can be 
improved. 

[0031 ] Next, a process up to the start of an image rec- 
ognition process will be described. A combination of the 
camera monitor position, the pan, tilt, and focus posi- 
tions for each monitor position is stored in accordance 
with preset information, and the focus position of the 
lens is changed. Fig. 3 is a flow chart illustrating the op- 
eration up to the start of an image recognition process. 
At Step 301 in accordance with preset information, the 
camera monitor position such as the pan and tilt posi- 
tions for each monitor position is set in accordance with 
preset camera position information . At Step 302 the lens 
focus position is set in accordance with information such 
as the view angle size corresponding to each preset 
monitor view angle and a lens focus position. 
[0032] Thereafter, at Step 303 an AF operation timer 
is started. At Step 304 the AF operation starts. If the 
background image is a flat image with less edges such 
as a wall, it takes a long time to complete the AF oper- 
ation, and at the worst the AF operation may not be com- 
pleted. In such a case, the time-out of the timer occurs 
at Step 306. At Step 307 the focus position is again set 
to the preset lens focus position to move the focus po- 
sition to that of the background image. 
[0033] In this state, the image recognition process 
starts at Step 309. In this manner, a delay of the start of 
the image recognition process is suppressed. If the in- 
focus state is set in the predetermined time duration, 
then the image recognition process starts at Step 309. 
In this manner, the image recognition process can start 
and an optimum monitor image can be obtained. 
[0034] Next, the AF operation and the image recogni- 
tion process will be described. Fig. 4 shows an example 
of an image without any intruder. The AF focus position 
is at a background image 401 . A difference between an 
input monitor image and the preset background image 
is used for detecting an abnormal entrance. In this ex- 
ample, this difference is not detected and no abnormal 
entrance is detected. 

[0035] Fig. 5 shows an example of an image with an 
intruder near to the camera. Reference numeral 501 
represents the background image, and reference nu- 
meral 502 represents an intruder image. In this case, 
the AF focus position is at the intruder image 502. There- 
fore the background image 501 is an out-of-focus im- 
age. With the image recognition process, a difference 
between the preset background image and the input 
monitor image is used for detecting an abnormal en- 
trance. An area in which an abnormal entrance is to be 
detected by using the difference can be set as desired. 
If the whole display screen is set as such an area, a 
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difference between the intruder image 502 and the out- 
of-focus background image 501 appears so that an ab- 
normal entrance can be detected. In this case, a preci- 
sion of the image recognition process can be improved 
further if the out-of-focus background image is judged 5 
as the background image from the preset background 
image and if it is judged the intruder image is not con- 
tained in the preset background image. 
[0036] As above, if the background image is remote 
from the camera and the intruder is near to the camera, 10 
good image recognition can be performed by executing 
the AF operation. 

[0037] Fig. 6 shows an example of an image with an 
intruder remote from the camera. Reference numeral 
601 represents the background image and reference '5 
numeral 602 represents an intruder image. In this case, 
the AF focus position is approximately at the back- 
ground image. A difference between the preset back- 
ground image and the intruder image appears. 
[0038] As described so far, according to the present 20 
invention, even if the camera is moved to a different po- 
sition or it takes a different view angle during multi-point 
monitoring, the AF operation is completed after the pre- 
set focus position adjustment or after the AF operation. 
Accordingly, the image recognition can be performed by 25 
using a stable monitor image, and an abnormal entrance 
can be detected reliably. Furthermore, since the preset 
focus position adjustment and the AF operation are 
combined, the focusing time can be shortened, the start 
and end of the image recognition process can be made 30 
earlier and the abnormal entrance detection precision 
can be improved. 



Claims 35 

1 - A multi-point monitor camera system comprising: 

a pan/tilt driver unit (103) for changing a cam- 
era monitor position to monitor a plurality of . 40 
monitor positions; 

a focusing mechanism for changing a focus po- 
sition of a lens to adjust a focal point; 
automatic focusing means; and 
image recognition means (1 02) for detecting a *5 
change amount of a monitor image input via a 
camera lens and detecting an abnormal en- 
trance of an object, 

wherein the abnormal entrance at the plurality so 
of monitor positions is detected by making said im- 
age recognition means start an image recognition 
operation after an automatic focusing operation by 
said automatic focusing means is completed. 

55 

2. A multi-point monitor camera system comprising: 
a pan/tilt driver unit (103) for changing a cam- 



era monitor position to monitor a plurality of 
monitor positions; 

a focusing mechanism for changing a focus po- 
sition of a lens to adjust a focal point; 
image recognition means (102) for detecting a 
change amount of a monitor image input via a 
camera lens and detecting an abnormal en- 
trance of an object; and 
preset means (105) for storing a combination 
of pan , tilt and focus positions for each monitor 
position and a background image correspond- 
ing to the combination, 

wherein the monitor position for the camera is 
changed in accordance with preset information 
stored in said preset means and the abnormal en- 
trance at the plurality of monitor positions is detect- 
ed by making said image recognition means start 
an image recognition operation after completion of 
commanding to change the monitor position to a 
preset position or after stop at the preset position. 

3. A multi-point monitor camera system according to 
claim 2, further comprising automatic focusing 
means 

wherein after completion of changing to a preset po- 
sition or after stop at the preset position, said auto- 
matic focusing means executes an automatic focus- 
ing operation for a monitor object, and thereafter 
said image recognition means starts an image rec- 
ognition operation to detect the abnormal entrance 
at the plurality of monitor positions. 

4. A multi-point monitor camera system according to 
claim 2, further comprising automatic focusing 
means 

wherein after completion of commanding to change 
the monitor position to a preset position or afterstop 
at the preset position, said automatic focusing 
means executes an automatic focusing operation 
for a monitor object, thereafter said image recogni- 
tion means starts an image recognition operation, 
at the same time when the automatic focusing op- 
eration is executed, a timer is activated, if an in-fo- 
cus state does not enter after a predetermined time 
lapse, the focus position is forcibly set to the preset 
position, and thereafter said image recognition 
means starts the image recognition operation to de- 
tect the abnormal entrance at the plurality of monitor 
positions. 

5. A multi-point monitor camera system comprising: 

a focusing mechanism for changing a focus po- 
sition of a lens; 
automatic focusing means; 
a zoom mechanism for changing a camera 
monitor view angle to monitor a plurality of mon- 
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itor positions, at least two monitor positions 
having different view angles; 
storage means for storing a background image 
having at least one or more view angles; and 
image recognition means (1 02) for detecting a 5 
change amount of a monitor image input via a 
camera lens and detecting an abnormal en- 
trance of an object, 

wherein the abnormal entrance at the plurality 10 
of monitor positions is detected by making said im- 
age recognition means start an image recognition 
operation after an operation by said zoom mecha- 
nism or after completion of an operation by said au- 
tomatic focusing means. is 

6. A multi-point monitor camera system comprising: 

a focusing mechanism for changing a focus po- 
sition of a lens; 20 
a zoom mechanism for changing a camera 
monitor view angle to monitor a plurality of mon- 
itor positions, at least two monitor positions 
having different view angles; 

preset means (1 05) for storing a zoom position 25 
for each monitor angle and a corresponding 
background image; and 

image recognition means (102) for detecting a 
change amount of a monitor image input via a 
camera lens and detecting an abnormal en- 30 
trance of an object, 

wherein the camera monitor view angle is 
changed in accordance with preset position infor- 
mation stored in said preset means and the abnor- 35 
mal entrance at the plurality of monitor positions is 
detected by making said image recognition means 
start an image recognition operation after comple- 
tion of switching to a preset position or after com- 
pletion of an operation by said zoom mechanism. *o 

7. A multi-point monitor camera system according to 
claim 6, further comprising automatic focusing 
means 

wherein alter completion of changing to the present ^ 
position or after completion of an operation by said 
zoom mechanism, said automatic focusing means 
executes an automatic focusing operation for a 
monitor object, and thereafter said image recogni- 
tion means starts an image recognition operation to 50 
detect the abnormal entrance at the plurality of 
monitor positions. 

8. A multi-point monitor camera system according to 
claim 6, further comprising automatic focusing 55 
means 

wherein after completion of changing to the preset 
position or after completion of an operation by said 



zoom mechanism, said automatic focusing means 
executes an automatic focusing operation for a 
monitor object, thereafter said image recognition 
means starts an image recognition operation, at the 
same time when the automatic focusing operation 
is executed, a timer is activated, if an in-focus state 
does not enter after a predetermined time lapse, the 
focus position is forcibly set to the preset position, 
and thereafter said image recognition means starts 
the image recognition operation to detect the abnor- 
mal entrance at the plurality of monitor positions. 
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(54) Multi-point camera system 



(57) A multi-point surveillance camera system uses 
a combination of an image recognition unit (1 02) a zoom 
camera (101) equipped with a pan/tilt driver unit (103) 
for changing a camera monitor position, to monitor a plu- 
rality of monitor positions. The monitor position is 
changed by using the pan/tilt unit, and an image recog- 
nition unit starts an image recognition process to reliably 



detect an abnormal entrance at the plurality of monitor 
positions, after a focus position adjustment by preset in- 
formation (after completion of commanding to change 
the monitor position to a preset position or stop at the 
preset position), after an automatic focusing operation, 
or after a combination of the focus position adjustment 
by the preset information and the automatic focusing op- 
eration. 
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